Allow external structs. #2798

mikebenfield · 2025-07-07T01:58:56Z

Specifically,

Introduce a PlaintextType::ExternalStruct, and allow referring to such a type in .aleo code by a Locator.
Allow casting to external struct types.
Let two struct types be considered the same type as long as they are structurally the same and have the same local names. This funky ruleset is effectively here for backwards compatibility. In more detail:

/// Equivalence of structs means they have the same local names (regardless of whether
/// they're local or external), and their members have the same names and equivalent
/// types in the same order, recursively.
///
/// Equivalence of arrays means they have the same length and their element types are
/// equivalent.
///
/// This definition of equivalence was chosen to balance these concerns:
///
/// 1. All programs from before the existence of external structs will continue to work -
/// thus it's necessary for a struct created from another program to be considered equivalent
/// to a local one with the same name and structure, as in practice that was the behavior.
/// 2. We don't want to allow a fork. Thus we do need to check names, not just structural
/// equivalence - otherwise we could get a program deployable to a node which is using
/// this check, but not deployable to a node running an earlier SnarkVM.

Also note this other usecase from a user:

currently we got in an issue where one mainnet deployed contract imports tokenRegistry, and the contract we are writing for A imports that contract and A contract. but, A contract and tokenRegistryContract have same struct names so we cannot compile that.If we cannot change struct name in A contract, we need to wait for snarkVM update, just to compile the contracts

Specifically, 1. Introduce a PlaintextType::ExternalStruct, and allow referring to such a type in .aleo code by a Locator. 2. Allow casting to external struct types. 3. Let two struct types be considered the same type as long as they are structurally the same.

mikebenfield · 2025-07-23T16:23:32Z

Rebased.

vicsn

Can you add a check in check_transaction that deployments using an external struct are only enabled from ConsensusVersion::V10 onwards. You can put in dummy heights for the new consensus version.

And while you're at it, consider refactoring ConsensusVersion-related checks into a separate function called by check_transaction?

mikebenfield · 2025-07-25T18:38:23Z

I moved some of the consensus version checks into a new function, but some others are scattered throughout check_transaction so this may or may not be what you had in mind.

vicsn

Very nice.

I would appreciate a short walk through synthesizer/program/src/logic/instruction/operation/cast.rs during the next Leo Weekly.

I'll defer to @d0cd about the usage and implementation of types_structurally_equivalent and whether he seed potential impending issues related to dynamic dispatch.

synthesizer/tests/tests/parser/program/external_struct.aleo

synthesizer/src/vm/verify.rs

synthesizer/program/src/function/mod.rs

synthesizer/program/src/logic/instruction/mod.rs

console/program/src/data_types/register_type/mod.rs

synthesizer/program/src/logic/instruction/operation/call.rs

synthesizer/program/src/parse.rs

synthesizer/tests/tests/process/execute/external_struct.aleo

synthesizer/program/src/logic/instruction/operation/cast.rs

Added a failing test using a non-existant external struct.

mikebenfield · 2025-07-29T16:56:11Z

Here's some updates.

You also asked me to walk you through the changes in cast.rs, which I didn't do yesterday. It's actually pretty straightforward so I'll just list what I've done here. The point is just to pull out the code that previously just applied to Plaintext::Struct into separate functions so it can be called when finding either Plaintext::Struct or Plaintext::ExternalStruct:

Previously the code to actually carry out casting to structs was carried out in two places: the cast_to_struct function handled console values, while the circuit values were handled inline in the execute function. I factored out that common code to a cast_to_struct_common macro, used that to define separate execute_cast_to_struct and evaluate_cast_to_struct functions, and then called those functions when evaluating or executing where the destination type is Struct or ExternalStruct.
Previously the code did a bunch of checks inline in output_types when the dest type what a struct. I pulled those checks into a closure called struct_checks, and now call that when matching either a Struct or an ExternalStruct.

vicsn · 2025-07-30T14:01:54Z

Can you make a PR to add clarification to the docs:

https://github.com/AleoNet/welcome/blob/master/documentation/guides/aleo/04_opcodes.md?plain=1#L218
https://github.com/AleoNet/welcome/blob/master/documentation/guides/aleo/04_opcodes.md?plain=1#L251
https://github.com/AleoNet/welcome/blob/master/documentation/guides/aleo/04_opcodes.md?plain=1#L284 - it may be best to just refer back to the assert.eq documentation
https://github.com/AleoNet/welcome/blob/master/documentation/guides/aleo/04_opcodes.md?plain=1#L314
https://github.com/AleoNet/welcome/blob/master/documentation/guides/aleo/04_opcodes.md?plain=1#L435 - you mention in a comment above that we can't cast.lossy to a struct
https://github.com/AleoNet/welcome/blob/master/documentation/guides/aleo/04_opcodes.md?plain=1#L1246 - again may be best to just refer back to earlier documentation
https://github.com/AleoNet/welcome/blob/master/documentation/guides/aleo/04_opcodes.md?plain=1#L1279

vicsn · 2025-07-30T14:39:17Z

Can you use this to confirm that the R1CS for all known deployments remains unchanged? https://github.com/ProvableHQ/aleo-program-regressions/tree/master/r1cs-hasher

EDIT: It's been a while since we touched it, so may need some minor refactoring. Specifically, I see it is unclear which deployments have been used for the previous hash, I assume the first 211 of either canary/testnet/mainnet because the hash file name ends with an inexplicable _211. I'd advise computing a hash from scratch with all programs, using staging and struct-namespace snarkVM versions.

mikebenfield · 2025-07-30T14:59:39Z

Can you make a PR to add clarification to the docs:

Sure.

mikebenfield · 2025-07-30T15:33:07Z

Can you use this to confirm that the R1CS for all known deployments remains unchanged?

I can't get this to work on recent snarkvm commits. Using snarkvm staging branch (after making a couple changes to main.rs to account for snarkvm API changes):

% RUST_BACKTRACE=1 ~/.cargo/bin/r1cs-hasher programs/mainnet external-struct-mainnet
Processed 282 deployments

Initializing 'TestRng' with seed '0'


thread '<unnamed>' panicked at /Users/michaelbenfield/Code/snarkVM/circuit/environment/src/circuit.rs:293:9:
Surpassed the variable limit (41859)

UPDATE written by Victor: the tests passed now.

mikebenfield · 2025-07-30T15:48:00Z

Regarding the docs about all the commands/instructions that compare equality of structs. Here's the actual behavior (both before and after this PR; it's unchanged):

If you're trying to compare two struct values and their types have different names, the program is rejected and snarkvm doesn't even try to run it;
If you're trying to compare two struct values and their types have the same name but different structure (like different member names or types), the program begins running but there will be a runtime error (not a failed constraint);
If you're trying to compare two struct values and their types have the same name and structure, then the comparison will work as expected.

I think this is not really behavior we want users to think about, but instead just want to users to only try to compare two struct values of the same type. For that reason I'm inclined to leave the documentation of those instructions as is, but let me know if you feel otherwise.

Change tag for ArrayType element type. Introduce and use finalize_types_structurally_equivalent. Use types_structurally_equivalent in synthesizer/process/src/stack/finalize_types/matches.rs `ProgramID` with backticks in comments.

console/network/src/consensus_heights.rs

mikebenfield · 2025-09-15T19:47:02Z

@d0cd I've checked your questions again; I added a couple comments but I think the code as of the last push had addressed all the issues you raised. Let me know what you think.

d0cd · 2025-09-19T13:00:19Z

synthesizer/src/vm/verify.rs

-                    );
-                }
+
+                let stack = self.process.read().get_stack(deployment.program_id())?;


Is the stack guaranteed to exist at this point?

Ooff good catch. No it's not. And we don't even need it, we just read the program from it. We should add some integration tests in a new test_v11.rs file. You can copy behaviour from test_v10.rs and test_v9.rs: e.g. simply create a transaction with external struct before the V11 height, should fail to verify, after the V11 height, should succeed to verify.

Good grief I'm a mess; sorry. I actually already fixed this locally before I even pushed this but somehow pushed the wrong commit. I've now pushed the fix, along with the changes to comments. I'll add the test_v11.rs shortly.

d0cd · 2025-09-19T13:01:01Z

synthesizer/src/vm/verify.rs

+    // If the `CONSENSUS_VERSION` is greater than or equal to `V9`, then verify that:
+    //   - the program checksum is present in the deployment
+    //   - the program owner is present in the deployment
+    // If the `CONSENSUS_VERSION` is less than `V10`, then verify that:


Suggested change

// If the `CONSENSUS_VERSION` is less than `V10`, then verify that:

// If the `CONSENSUS_VERSION` is less than `V11`, then verify that:

d0cd · 2025-09-19T13:02:26Z

synthesizer/src/vm/verify.rs

+    //   - the program checksum is present in the deployment
+    //   - the program owner is present in the deployment
+    // If the `CONSENSUS_VERSION` is less than `V10`, then verify that:
+    //   - the program does not use the external struct syntax `some_program.aleo/StructT`


Suggested change

// - the program does not use the external struct syntax `some_program.aleo/StructT`

// - the program does not use the external struct syntax `some_program.aleo/StructT`

// - the program's mappings do not use non-existent structs.

vicsn · 2025-09-19T15:46:58Z

synthesizer/program/src/logic/instruction/operation/is.rs

        }
        // Ensure the operands are of the same type.
-        if input_types[0] != input_types[1] {
+        if !register_types_structurally_equivalent(stack, &input_types[0], stack, &input_types[1])? {


Forking risk, transaction validity changed

Also change type equivalence to require structs have the same name.

synthesizer/tests/tests/process/execute/external_struct_fail.aleo

synthesizer/src/vm/verify.rs

- external_struct_fail_name test - `plaintext_exists` moved to `Process` as `mapping_types_exist`.

vicsn · 2025-09-26T12:57:21Z

@mikebenfield one of the synthesizer tests halted with a mysterious VM safely halted at /home/circleci/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/itertools-0.14.0/src/zip_eq_impl.rs:50:17: itertools: .zip_eq() reached end of one iterator before the other. Would be great if you can try to reproduce.

Note that a small improvement to test_vm_execute_and_finalize tests along the way may both speed up your debugging and be useful for the future:

would there have been any way for this custom test runner to log which test failed on this particular error, so that for reproducing you'd only have to run a single one?
speaking of running a single test, it is currently not possible or easy to see how to choose a single sub-test to run in test_vm_execute_and_finalize

mikebenfield · 2025-09-30T04:54:03Z

@vicsn Alright I fixed that test but unfortunately didn't enhance the test function.

There's now a semver check failing, but I don't think that is related to this PR.

vicsn · 2025-11-05T15:31:50Z

@mohammadfawaz can records contain structs and/or external structs? Do we have tests for this in place? CC @Antonio95 for his work on dynamic dispatch

mikebenfield requested review from d0cd and vicsn July 7, 2025 01:58

mikebenfield force-pushed the struct-namespace branch from 2eb1e7b to 0b0aac0 Compare July 7, 2025 02:09

mikebenfield marked this pull request as draft July 7, 2025 15:56

mikebenfield marked this pull request as ready for review July 7, 2025 15:57

mikebenfield force-pushed the struct-namespace branch from 0b0aac0 to 410d4a3 Compare July 7, 2025 17:35

mikebenfield force-pushed the workspace-dependencies branch from dd47c44 to f6e272f Compare July 7, 2025 22:34

mikebenfield force-pushed the struct-namespace branch from 410d4a3 to 687192d Compare July 7, 2025 22:34

mikebenfield force-pushed the workspace-dependencies branch from f6e272f to 5ec993c Compare July 8, 2025 02:24

mikebenfield force-pushed the struct-namespace branch from 687192d to b891a9d Compare July 8, 2025 02:24

Base automatically changed from workspace-dependencies to staging July 12, 2025 09:55

Allow external structs.

558625f

Specifically, 1. Introduce a PlaintextType::ExternalStruct, and allow referring to such a type in .aleo code by a Locator. 2. Allow casting to external struct types. 3. Let two struct types be considered the same type as long as they are structurally the same.

mikebenfield force-pushed the struct-namespace branch from b891a9d to 558625f Compare July 23, 2025 16:23

vicsn reviewed Jul 25, 2025

View reviewed changes

mikebenfield added 2 commits July 25, 2025 09:48

Merge branch 'staging' into struct-namespace

46bf2c1

Disallow deployments using external struct before ConsensusVersion::V10.

e3601da

vicsn requested changes Jul 25, 2025

View reviewed changes

mikebenfield added 2 commits July 29, 2025 09:43

Added more varied uses of external structs in parsing test.

839f17d

Added a failing test using a non-existant external struct.

Merge branch 'staging' into struct-namespace

4112865

mikebenfield added 2 commits July 30, 2025 07:42

Fix external_struct_fail test

0085736

Merge branch 'staging' into struct-namespace

d7e0691

Fix tests

04b2a49

mikebenfield added 3 commits August 24, 2025 13:21

Address review comments.

0f4bb1b

Change tag for ArrayType element type. Introduce and use finalize_types_structurally_equivalent. Use types_structurally_equivalent in synthesizer/process/src/stack/finalize_types/matches.rs `ProgramID` with backticks in comments.

Test accessing a field of an external struct

71aef7f

clippy

e5f7e3b

vicsn reviewed Aug 26, 2025

View reviewed changes

console/network/src/consensus_heights.rs Outdated Show resolved Hide resolved

mikebenfield added 2 commits August 28, 2025 09:12

Disallow nonexistent structs in mappings

377e066

Merge branch 'staging' into struct-namespace

95364fc

mikebenfield requested a review from d0cd August 29, 2025 16:01

Merge branch 'staging' into struct-namespace

ee0e0ee

mikebenfield added 3 commits September 16, 2025 10:22

Move type checks for mapping types to check_transaction

c015c7d

Merge branch 'staging' into struct-namespace

505aecc

Fix mapping type check

7e02f20

d0cd reviewed Sep 19, 2025

View reviewed changes

comments

2985f4e

vicsn reviewed Sep 19, 2025

View reviewed changes

test_v11.rs

55038dc

Also change type equivalence to require structs have the same name.

vicsn reviewed Sep 24, 2025

View reviewed changes

synthesizer/tests/tests/process/execute/external_struct_fail.aleo Show resolved Hide resolved

vicsn reviewed Sep 24, 2025

View reviewed changes

synthesizer/src/vm/verify.rs Outdated Show resolved Hide resolved

mikebenfield added 2 commits September 25, 2025 12:07

Review comments.

1204947

- external_struct_fail_name test - `plaintext_exists` moved to `Process` as `mapping_types_exist`.

Merge branch 'staging' into struct-namespace

4348c3f

mikebenfield added 2 commits September 29, 2025 20:18

Fix external_struct_in_mapping test

b5826a2

Merge branch 'staging' into struct-namespace

96ee3a0

Mohammad Fawaz added 3 commits November 3, 2025 12:08

Merge branch 'staging' into struct-namespace

fd0779c

Fix a few things

7a16a80

Support serialize/deserialize and size_in_bits for external structs

985a954

	// If the `CONSENSUS_VERSION` is less than `V10`, then verify that:
	// If the `CONSENSUS_VERSION` is less than `V11`, then verify that:

	// - the program does not use the external struct syntax `some_program.aleo/StructT`
	// - the program does not use the external struct syntax `some_program.aleo/StructT`
	// - the program's mappings do not use non-existent structs.

Allow external structs. #2798

Are you sure you want to change the base?

Allow external structs. #2798

Uh oh!

Conversation

mikebenfield commented Jul 7, 2025 • edited by vicsn Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mikebenfield commented Jul 23, 2025

Uh oh!

vicsn left a comment

Choose a reason for hiding this comment

Uh oh!

mikebenfield commented Jul 25, 2025

Uh oh!

vicsn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mikebenfield commented Jul 29, 2025

Uh oh!

vicsn commented Jul 30, 2025

Uh oh!

vicsn commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mikebenfield commented Jul 30, 2025

Uh oh!

mikebenfield commented Jul 30, 2025 • edited by vicsn Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mikebenfield commented Jul 30, 2025

Uh oh!

Uh oh!

mikebenfield commented Sep 15, 2025

Uh oh!

d0cd Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

vicsn Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mikebenfield Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

d0cd Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

d0cd Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

vicsn Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

vicsn commented Sep 26, 2025

Uh oh!

mikebenfield commented Sep 30, 2025

Uh oh!

vicsn commented Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mikebenfield commented Jul 7, 2025 •

edited by vicsn

Loading

vicsn commented Jul 30, 2025 •

edited

Loading

mikebenfield commented Jul 30, 2025 •

edited by vicsn

Loading

vicsn Sep 19, 2025 •

edited

Loading

vicsn Sep 19, 2025 •

edited

Loading